feat: report ElasticCloud data tiers nodes roles #652

christophemorio · 2022-11-17T12:02:08Z

On ElasticCloud clusters, nodes have specific roles regarding data tier leveling:

data_hot
data_warm
data_cold
data_frozen

This PR add support for those roles.

curl -s http://localhost:9108/metrics | grep 'elasticsearch_nodes_roles'
# HELP elasticsearch_nodes_roles Node roles
# TYPE elasticsearch_nodes_roles gauge
elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="data_hot"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="ingest"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="data_hot"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="ingest"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.236.46",name="instance-0000000003",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.236.46",name="instance-0000000003",role="data_warm"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.238.200",name="instance-0000000019",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.238.200",name="instance-0000000019",role="master"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.205",name="instance-0000000021",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.205",name="instance-0000000021",role="data_frozen"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.47",name="instance-0000000022",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.47",name="instance-0000000022",role="data_cold"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.154",name="instance-0000000018",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.154",name="instance-0000000018",role="master"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.162",name="instance-0000000007",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.162",name="instance-0000000007",role="data_warm"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.245.96",name="instance-0000000020",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.245.96",name="instance-0000000020",role="master"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.246.94",name="instance-0000000010",role="client"} 1
elasticsearch_nodes_roles{cluster="xxxx",host="172.25.246.94",name="instance-0000000010",role="data_cold"} 1

Fixes #651

ariahi18

LGTM

sysadmind · 2022-11-27T19:44:10Z

collector/nodes.go

@@ -84,7 +88,9 @@ func createRoleMetric(role string) *nodeMetric {
 }

 var (
-	defaultNodeLabels               = []string{"cluster", "host", "name", "es_master_node", "es_data_node", "es_ingest_node", "es_client_node"}
+	defaultNodeLabels = []string{"cluster", "host", "name", "es_master_node", "es_data_node",


I think the format of the node metrics would need to change to add more roles. Adding all of these labels to the nodes metrics doesn't feel like a good idea to me. This variable appears to be referenced over 100 times in this file with most references being adding all of these labels to all of the metrics. A better approach may be to expose a new metric for node roles as a gauge with a 1 or 0 based on if the node has that role. I know that's a much different change to this and more involved, but I don't think it's sustainable to keep adding labels to these metrics.

Hi @sysadmind ,

It makes sens, I see the point of nodes labels multiplicity.
Then, I propose an approach by changing only outputs of elasticsearch_nodes_roles{} in the next commit.

Output updated

$ curl -s http://localhost:9108/metrics | grep 'elasticsearch_nodes_roles' # HELP elasticsearch_nodes_roles Node default roles # TYPE elasticsearch_nodes_roles gauge elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="data_content"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="data_hot"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="ingest"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.138.46",name="instance-0000000016",role="transform"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="data_content"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="data_hot"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="ingest"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.22.143.78",name="instance-0000000015",role="transform"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.236.46",name="instance-0000000003",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.236.46",name="instance-0000000003",role="data_warm"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.236.46",name="instance-0000000003",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.238.200",name="instance-0000000019",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.238.200",name="instance-0000000019",role="master"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.238.200",name="instance-0000000019",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.205",name="instance-0000000021",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.205",name="instance-0000000021",role="data_frozen"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.47",name="instance-0000000022",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.47",name="instance-0000000022",role="data_cold"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.241.47",name="instance-0000000022",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.154",name="instance-0000000018",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.154",name="instance-0000000018",role="master"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.154",name="instance-0000000018",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.162",name="instance-0000000007",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.162",name="instance-0000000007",role="data_warm"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.242.162",name="instance-0000000007",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.245.96",name="instance-0000000020",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.245.96",name="instance-0000000020",role="master"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.245.96",name="instance-0000000020",role="remote_cluster_client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.246.94",name="instance-0000000010",role="client"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.246.94",name="instance-0000000010",role="data_cold"} 1 elasticsearch_nodes_roles{cluster="xxxx",host="172.25.246.94",name="instance-0000000010",role="remote_cluster_client"} 1

Signed-off-by: Christophe MORIO <cmorio@talend.com>

eloo-abi · 2023-03-07T14:22:37Z

Hi, are there any news for this PR?

christophemorio · 2023-03-15T17:57:33Z

Hi @sysadmind , is it good for you ?

sysadmind

LGTM

BREAKING CHANGES: The flag `--es.cluster_settings` has been renamed to `--collector.clustersettings`. * [CHANGE] Rename --es.cluster_settings to --collector.clustersettings * [FEATURE] Add ILM metrics #513 * [ENHANCEMENT] Add ElasticCloud node roles to role label #652 * [ENHANCEMENT] Add ability to use AWS IAM role for authentication #653 * [ENHANCEMENT] Add metric for index replica count #483 * [BUGFIX] Set elasticsearch_clusterinfo_version_info guage to 1 #728 * [BUGFIX] Fix index field counts with nested fields #675 --------- Signed-off-by: Joe Adams <github@joeadams.io>

…y#652) * feat: additional elasticcloud roles --------- Signed-off-by: Christophe MORIO <cmorio@talend.com>

…community#652)" This reverts commit ad3c442.

christophemorio force-pushed the support-elasticcloud-node-roles branch from d11b559 to c8a1083 Compare November 17, 2022 13:10

ariahi18 approved these changes Nov 17, 2022

View reviewed changes

sysadmind reviewed Nov 27, 2022

View reviewed changes

christophemorio added 3 commits November 29, 2022 09:21

feat: additional elasticcloud roles

db005d5

Signed-off-by: Christophe MORIO <cmorio@talend.com>

feat: enhance only nodes_roles metrics with non default roles

00302dc

Signed-off-by: Christophe MORIO <cmorio@talend.com>

chore: revert undesired change

b6b8ff7

Signed-off-by: Christophe MORIO <cmorio@talend.com>

christophemorio force-pushed the support-elasticcloud-node-roles branch from a78f309 to b6b8ff7 Compare November 29, 2022 08:21

christophemorio requested a review from sysadmind November 30, 2022 09:37

hwu-talendbj approved these changes Dec 6, 2022

View reviewed changes

sysadmind approved these changes Mar 23, 2023

View reviewed changes

sysadmind merged commit 687e75e into prometheus-community:master Mar 23, 2023

christophemorio deleted the support-elasticcloud-node-roles branch April 4, 2023 12:38

jaimeyh added a commit to sysdiglabs/elasticsearch_exporter that referenced this pull request Jun 14, 2024

Revert "feat: report ElasticCloud data tiers nodes roles (prometheus-…

f907b12

…community#652)" This reverts commit ad3c442.

jaimeyh added a commit to sysdiglabs/elasticsearch_exporter that referenced this pull request Jun 14, 2024

Revert "feat: report ElasticCloud data tiers nodes roles (prometheus-…

97fe97c

…community#652)" This reverts commit ad3c442.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: report ElasticCloud data tiers nodes roles #652

feat: report ElasticCloud data tiers nodes roles #652

christophemorio commented Nov 17, 2022

ariahi18 left a comment

sysadmind Nov 27, 2022

christophemorio Nov 28, 2022

christophemorio Nov 28, 2022

hwu-talendbj Dec 6, 2022

eloo-abi commented Mar 7, 2023

christophemorio commented Mar 15, 2023

sysadmind left a comment

feat: report ElasticCloud data tiers nodes roles #652

feat: report ElasticCloud data tiers nodes roles #652

Conversation

christophemorio commented Nov 17, 2022

ariahi18 left a comment

Choose a reason for hiding this comment

sysadmind Nov 27, 2022

Choose a reason for hiding this comment

christophemorio Nov 28, 2022

Choose a reason for hiding this comment

christophemorio Nov 28, 2022

Choose a reason for hiding this comment

hwu-talendbj Dec 6, 2022

Choose a reason for hiding this comment

eloo-abi commented Mar 7, 2023

christophemorio commented Mar 15, 2023

sysadmind left a comment

Choose a reason for hiding this comment